Mobile secretary cloud application

ABSTRACT

The disclosure provides a method, system, and a software program product for assisting a user and/or managing tasks of the user, by a mobile secretary cloud application configured to operate in a mobile client device and cloud server network. The mobile secretary cloud application reads data from another software application and operates at least one of another application and a third application based on the read data. Further, Artificial intelligence is utilized by the mobile secretary cloud application for operating another application and the third application.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present disclosure claims benefit of and priority to co-pending U.S.patent application Ser. No. 17/505,189 filed Oct. 19, 2021, which is acontinuation of U.S. patent application Ser. No. 16/109,731 filed Aug.22, 2018, later issued on Nov. 23, 2021 as U.S. Pat. No. 11,184,471,which claims the benefit of U.S. Provisional Patent Application Ser. No.62/590,291 filed Nov. 23, 2017, each of which are incorporated herein byreference in their entireties.

TECHNICAL FIELD

Aspects of the invention relate to managing tasks of a user through asmartphone and more particularly to utilizing artificial intelligence inmanaging tasks of the user.

BACKGROUND

Modern smartphones carry numerous software applications for performingdifferent specialized tasks. For example, a calendar application keepsrecord of activities scheduled for particular dates, an emailingapplication manages emails received and sent by a user, and imagegallery manages images and videos captured using the smartphone.Similarly, there are numerous other applications for managing differentdata relevant to the user.

Many times, it becomes a tedious task for the user to manage suchnumerous applications. For example, the user may need to refer to afirst application and may need to manage a second application based onthe first application. For example, the user may need to access theemailing application for accessing an email comprising details of ameeting scheduled for the user. Based on retrieved details of themeeting, the user may need to revert with an acceptance of the meeting,set an alarm or a reminder for the date of meeting, set a navigationroute for reaching a venue of meeting, or book flight/train/bus forreaching the venue of meeting. Similarly, the user may need to manageseveral activities using different applications installed on hissmartphone, based on information retrieved from one application. Presentstate of art merely comprises management of tasks based on user'sinstructions, either provided manually or via speech.

For example Siri software in the iPhone may take verbal instructionsfrom a user to operate the iPhone.

For example, US20130110519 describes a method to operate an intelligentdigital assistant for assisting a user in task management. Theintelligent digital assistant stores a plurality of predefined domainsindicative of areas of services offered. From a user's request (speechinput), one or more words are derived. Thereafter, a match is identifiedbetween the words derived from the user's request and words associatedwith the plurality of predefined domains. Based on a degree of match, arelevant domain for the user request is identified. Successively, a taskis executed based on steps specified in a task flow associated with therelevant domain and task parameters derived from the user's request.

WO2014197635A2 describes an intelligent automated assistant system. Theintelligent automated assistant system receives a search request from auser. The search request includes at least a speech input and one ormore search criteria for identifying reservable items. In response toreceiving the search request, a plurality of search results,corresponding to reservable items, are presented to the user.Successively, a reservation request is received for one search resultamongst the plurality of search results. In response to the reservationrequest, a reservation procedure is executed to reserve a respectivereservable item.

US20160155442 describes a digital personal assistant configured toperform tasks based on user instructions. Firstly, a user input isreceived, in form of a voice command or a text, for performing a task.Speech recognition is performed on the user input provided as voicecommand. Thereafter, the task is recognized and a registered actionprovider to provide with the task details. Successively, instructionsare sent to the particular action provider for performing the task.Machine learning is used within the process for determining userpreferences, for example if the user selects one action provider 5 timesin a row, on the 6^(th) time that provider is set as the default option.

Thus, the prior art features a number of simplistic workflows andautomations, and even discloses very simple steps for learning thepreference of the user in terms of one criterion. Individual automationsof tasks, however, are not capable of reducing the wholisticadministrative burden of the smartphone user in a significant way.

SUMMARY

It is an object of the invention to address and improve aforementioneddeficiencies in above discussed prior art(s). The object of theinvention is to provide an electronic artificial intelligence softwarebased secretary to every user, at a minimal fraction of the cost that itcosts to hire a human secretary.

The central philosophy of the invention is not to guess what the userwould do, given a certain input into the device. Ordinary consumers arenotoriously inefficient in managing their time. Successful people becomesuccessful partly because they have hired and employ successful personalassistants (PAs) and secretaries. The central philosophy of theinvention is how a professional secretary would handle the input, tomaximize the benefit to the user. In order to achieve this aim,technical software embodiments are produced in the following.

It is an object of the invention to assist a user and/or manageadministrative tasks of the user. Such assistance is provided to theuser by a mobile secretary cloud application configured to operate in amobile client device and cloud server network. Some embodiments of theinvention are described using systems, methods, and software programproducts, as explained below.

A method for assisting a user and/or managing tasks of the user, by amobile secretary cloud application configured to operate in a mobileclient device and cloud server network, is in accordance with someembodiments of the invention and characterized by the following steps:

-   -   the mobile client device is a smartphone connected to at least        one cloud server network via a wireless communication network,    -   a data analyzing module of the mobile secretary cloud        application reads data from at least one other application,    -   a data modifying module of the mobile secretary cloud        application operates the at least one other application, based        on data read from the at least one other application, and    -   an artificial intelligence module of the mobile secretary cloud        application uses artificial intelligence to operate the at least        one other application for assisting a user or to use the at        least one other application independently for performing at        least one task for the user.

The mobile secretary cloud application may replicate secretaryfunctions. In one case, the at least one other application is an emailor messaging application. The mobile secretary cloud applicationanalyses messages and automatically sorts the messages to differentfolders. Further, the mobile secretary cloud application analysesmessages and selects an indication sound automatically based on contentsof the messages and sender and/or recipients of the messages. In anothercase, the at least one other application is a telephony application, andthe mobile secretary cloud application makes phone calls on behalf ofthe user, automatically answers to messages, and prompts queries onbehalf of the user, using a synthesized voice on a phone line. In yetanother case, the at least one other application is a map application,and the mobile secretary cloud application provides a route calculatorthat provides a combined route of driving and walking based on emails,calendar entries, and location search. In a preferred embodiment thesecretary application uses artificial intelligence to determine whatwould be the optimum way of conducting the three activitiesindividually. Then, further in a preferred embodiment, the secretaryapplication performs a optimum total fit of all of the combinedactivities for the user's day. This may involve for example doing theautomated phone calls during the lunch break of the user, as an eatinguser is less likely to talk. The secretary application will propose adraft email response to a received email, selecting a short matter witha short response for a 10 minute break in the schedule of the user. Incontrast for an entire silent afternoon in the office, the secretaryapplication will present a 30 page project plan for inspection, as thesecretary application knows that now is the convenient time for that.Similarly, for travel, the shortest route that allows the execution ofthe most urgent matters is selected in accordance with some embodimentsof the invention.

A method for assisting a user and/or managing tasks of the user, by amobile secretary cloud application configured to operate in a mobileclient device and cloud server network is in accordance with someembodiments of the invention and characterized by the following steps:

-   -   the mobile client device is a smartphone connected to at least        one cloud server network via a wireless communication network,    -   a data analyzing module of the mobile secretary cloud        application is configured to read data from at least one other        application,    -   a data modifying module of the mobile secretary cloud        application is configured to operate a third application,        different from the at least one other application, based on data        read from the at least one other application,    -   an artificial intelligence module of the mobile secretary cloud        application uses artificial intelligence to operate the third        application for assisting a user or to use the third application        independently for performing at least one task for the user.

In one embodiment, the third application is an email or messagingapplication. The mobile secretary cloud application analyses messagesand automatically sorts messages to different folders based on dataderived from a calendar application or an Internet browser. Further, themobile secretary cloud application analyses messages and selects anindication sound automatically based on contents of the messages andsender and/or recipients of the messages, using a text analyzerapplication as the at least one other application.

In another embodiment, the third application is a telephony application,and the mobile secretary cloud application makes phone calls on behalfof the user, automatically answers to messages on behalf of the user,and prompts queries to the user. The mobile secretary cloud applicationperforms such functions using a synthesized voice on the phone line foroutgoing voice and using a voice recognition software application forincoming voice as the at least one other application.

In yet another embodiment, third application is a map application, andthe mobile secretary cloud application provides a route calculator forproviding a combined route of driving and walking based on data from theat least one other application, which data may comprise emails, calendarentries, search application, and Internet browser.

A mobile client device configured to execute a mobile secretary cloudapplication, wherein the mobile client device is a smartphone connectedto at least one cloud server network via a wireless communicationnetwork, and the mobile client device comprises:

-   -   a processor; and    -   a memory connected to the processor, wherein the processor is        configured to execute the mobile secretary cloud application to:        -   read data, using a data analyzing module, from at least one            other application;        -   operate, using a data modifying module, the at least one            other application based on data read from the at least one            other application; and        -   operate, using an artificial intelligence module, the at            least one other application for assisting a user or to use            the at least one other application independently for            performing at least one task for the user.

A mobile client device configured to execute a mobile secretary cloudapplication, wherein the mobile client device is a smartphone connectedto at least one cloud server network via a wireless communicationnetwork, and the mobile client device comprises:

-   -   a processor; and    -   a memory connected to the processor, wherein the processor is        configured to execute the mobile secretary cloud application to:        -   read data, using a data analyzing module, from at least one            other application;        -   operate, using a data modifying module, a third application,            different from the at least one other application, based on            data read from the at least one other application; and        -   operate, using an artificial intelligence module, the third            application for assisting a user or to use the third            application independently for performing at least one task            for the user.

A software program product stored in a memory medium for assisting auser and/or managing tasks of the user, by a mobile secretary cloudapplication, characterized by the following steps:

-   -   the software program product reads data from at least one other        application;    -   the software program product operates the at least one other        application based on data read from the at least one other        application; and    -   the software program product utilizes artificial intelligence to        operate the at least one other application for assisting the        user or to use the at least one other application independently        for performing at least one task for the user.

A software program product stored in a memory medium for assisting auser and/or managing tasks of the user, by a mobile secretary cloudapplication, characterized by the following steps:

-   -   the software program product reads data from at least one other        application;    -   the software program product operates a third application,        different from the at least one other application, based on data        read from the at least one other application; and    -   the software program product utilizes artificial intelligence to        operate the third application for assisting a user or to use the        third application independently for performing at least one task        for the user.

The best mode of the invention is considered to be a mobile secretaryapplication that is connected to the cloud servers. In the best mode,the cloud network continuously collects and updates self-learning filesfor a number of secretarial tasks, and makes them available to themobile secretary applications on the mobile client terminals. In thebest mode, the mobile secretary application and the cloud network firstdetermine what are the correct draft responses to each incoming taskindividually. Once the individual responses or actions to tasks havebeen determined, the agenda with which these responses or actions aremost efficiently completed with the user in a unit of time, for examplea day or a week, is determined. Once the agenda is determined, thesecretary application and cloud network guide the user through theresponses and activities in accordance with the determined agenda. Somesecretarial activities are also operated on a continuous and instantbasis, for example the playing of indication sounds for arrivingmessages, in the best mode.

The best mode can be applied to a number of uses, some of which mayoperate independently of the cloud network for long periods. Forexample, in the best mode, the selection of an indication sound isautomatically based on contents of the messages and sender and/orrecipients of the messages. In one case, AI is utilized to processcontents of the messages and thereby select an indication sound based oncontents of the messages. For example, the mobile client device of theuser may receive a pleasant message, stating “You did an incrediblejob.” Similarly, the user may receive other pleasant messages, such as“Thank you for your kind support,” “I am glad to hear about yourpromotion,” and “love you my son.” The email and message processingmodule 510, utilizing AI, may learn upon several such messages, and maydetermine that such messages convey pleasant gestures. The email andmessage processing module 510 may learn based on processing of wordspresent in the emails and messages. For example, in present case, thewords “incredible,” “thank you,” “kind,” “glad,” “promotion,” and “love”infer pleasant gestures from a sender, and the email and messageprocessing module 510 may learn such meaning. Based on such learning,the email and message processing module 510 may play uplifting orromantic audio, respectively, as the indication sound.

In contrast, when the mobile client device of the user may receive aharsh message, stating “poor job.” Similarly, the user may receive otherharsh messages, such as “you've passed the deadline,” “I won't toleratesuch careless attitude,” and “hate you.” The email and messageprocessing module 510, utilizing AI, may learn upon several suchmessages, and may determine that such messages convey harsh gestures.The email and message processing module 510 may learn based onprocessing of words present in the emails and messages. For example, inpresent case, the words “poor,” “passed deadline,” “won't tolerate,” and“hate” infer harsh gestures from a sender, and the email and messageprocessing module 510 may learn such meaning. Based on such learning,the email and message processing module 510 may play a loud beep orwarning sound as the indication sound.

Further, the email and message processing module 510 may also derivemeanings based on analysis of smileys/emoticons and special characterspresent in the emails and messages. The special characters may include“!!,” “?,” and other known special characters. For example, the user mayreceive a message stating “you again!!” or “what have you done?.” Theemail and message processing module 510 may learn that harsh gesturesare implied by such messages. Based on such learning, the email andmessage processing module 510 may play the loud beep or warning sound asthe indication sound. The indication sound selection embodiment can beimplemented as a cloud server—client device configuration, but also inthe client device only configuration as it requires few or no updatesfrom the network. In an alternative implementation of the best mode, thesubject line of the email or message or some content of the message isread out loud to the user, so that the user gets a quick indication ofthe contents without having to access the message itself via the screen.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an embodiment of a flow chart showing a method ofimplementation of a mobile secretary cloud application for operatinganother software application, according to an embodiment.

FIG. 1B illustrates an embodiment of a flow chart showing a method ofimplementation of a mobile secretary cloud application for operating athird software application, using another software application,according to an embodiment.

FIG. 2A illustrates an embodiment of a block diagram of a system forexecuting a mobile secretary cloud application, according to anembodiment.

FIG. 2B illustrates a machine learning embodiment where artificialintelligence is used to distinguish legitimate business emails from spamemails and other emails.

FIG. 3A illustrates an embodiment of a user interface showing a mobilesecretary cloud application operating another software application,according to an embodiment.

FIG. 3B illustrates an embodiment of a user interface showing a mobilesecretary cloud application utilizing another software application foroperating a third software application, according to an embodiment.

FIG. 4A illustrates an embodiment of a flow chart showing a method ofimplementation of a mobile secretary cloud application for operatingseveral other software applications, according to an embodiment.

FIG. 4B illustrates an embodiment of a flow chart showing a method ofimplementation of a mobile secretary cloud application for operatingseveral third software applications, using another software application,according to an embodiment.

FIG. 5 illustrates an embodiment of a block diagram of a systemexecuting a mobile secretary cloud application.

FIG. 6A illustrates an embodiment of a user interface showing a mobilesecretary cloud application operating an email/messaging application.

FIG. 6B illustrates an embodiment of a user interface showing a mobilesecretary cloud application operating an email/messaging application.

FIG. 6C illustrates an embodiment of a user interface showing a mobilesecretary cloud application operating a telephony application.

FIG. 6D illustrates an embodiment of a user interface showing a mobilesecretary cloud application operating a map application.

Some embodiments of the invention are described in the dependent claims.

DETAILED DESCRIPTION

The present disclosure provides a mobile client device, a method, and asoftware program product for assisting a user and/or managing tasks ofthe user. The tasks may be managed by a mobile secretary cloudapplication accessing an application installed in a mobile client deviceof the user. The mobile secretary cloud application may be availableover Google Play™ store for downloading over Android smartphones andover App Store™ for downloading over iOS™ smartphones. The mobilesecretary cloud application may either be available for free or as apaid application.

The mobile secretary cloud application is henceforth explained toutilize Artificial Intelligence for several purposes. Known machinelearning tools/deep learning frameworks may be utilized with or withoutmodifications. A few such known machine learning tools comprise Caffe™,Api.ai™, TensorFlow™, Mahout™, OpenNN™, H20™, MLlib™, NuPIC™ OpenCyc™,Oryx 2™, PredictionIO™, SystemML™, TensorFlow™, and Torch™.

FIG. 1A demonstrates an embodiment 10 of a method of implementation of amobile secretary cloud application for operating another softwareapplication, as a flow diagram. The method could be implemented in asystem identical or similar to embodiment 20 and 50 in FIG. 2 and FIG. 5, for example. The end-user of the method could use a user interfaceidentical or similar to that disclosed with embodiment 30, 31, 60, 61,62, and 63 in FIG. 3A, FIG. 3B, FIG. 6A, FIG. 6B, FIG. 6C, and FIG. 6D.

FIG. 2 illustrates an embodiment 20 of a block diagram of a clientsystem 200 executing the mobile secretary cloud application. The systemcomprises interface(s) 202, processor 204, Graphical Processing Unit(GPU) 206, and memory 208. The memory 208 comprises a data analyzingmodule 210, a data modifying module 212, and an Artificial Intelligence(AI) module 214. Different phases of FIG. 1A and FIG. 1B will now beexplained in conjunction with modules of FIG. 2 .

Interface(s) 202 are used to interact with or program the system 200.The interface(s) 202 may either be a Command Line Interface (CLI) or aGraphical User Interface (GUI) or both.

The processor 204 may refer to any one or more microprocessors, finitestate machines, computers, microcontrollers, digital signal processors,logic, a logic device, an electronic circuit, an application specificintegrated circuit (ASIC), a field-programmable gate array (FPGA), achip, etc., or any combination thereof, capable of executing computerprograms or a series of commands, instructions, or state transitions.The processor 204 may also be implemented as a processor set comprising,for example, a general purpose microprocessor and a math or graphicsco-processor. The processor 204 may be selected, for example, from theIntel® processors such as the Itanium® microprocessor or the Pentium®processors, Advanced Micro Devices (AMD®) processors such as the Athlon®processor, UltraSPARC® processors, microSPARC™ processors, HP®processors, International Business Machines (IBM®) processors such asthe PowerPC® microprocessor, the MIPS® reduced instruction set computer(RISC) processor of MIPS Technologies, Inc., RISC based computerprocessors of ARM Holdings, Motorola® processors, etc. The control unitdisclosed herein is not limited to employing the processor 204. Thecontrol unit may also employ a controller or a microcontroller and otherelectronics components.

The memory 208 includes a computer readable medium. A computer readablemedium may include volatile and/or non-volatile storage components, suchas optical, magnetic, organic or other memory or disc storage, which maybe integrated in whole or in part with a processor, such as processor204. Alternatively, all or part of the entire computer readable mediummay be remote from processor 204 and coupled to processor 204 byconnection mechanism and/or network cable. In addition to memory 208,there may be additional memories that may be coupled with the processor204 or the GPU (Graphics Processing Unit) 206.

In an embodiment, the system 200 is integrated with a cloud server 216,via a communication network 220. The cloud server may comprise a GPU 218or multiple GPUs. The communication network 220 used for thecommunication in some embodiments of the invention is the wireless orwireline Internet or the telephony network, which is typically acellular network such as UMTS—(Universal Mobile TelecommunicationSystem), GSM—(Global System for Mobile Telecommunications),GPRS—(General Packet Radio Service), CDMA—(Code Division MultipleAccess), 3G-, 4G-, Wi-Fi and/or WCDMA—(Wideband Code Division MultipleAccess) network.

In an example, the cloud server 216 may comprise a plurality of servers(not shown). In an example implementation, the cloud server 216 may beany type of a database server, a file server, a web server, anapplication server, etc., configured to store data related to the mobilesecretary cloud application and/or other applications. In anotherexample implementation, the cloud server 216 may comprise a plurality ofdatabases for storing the data files. The databases may be, for example,a structured query language (SQL) database, a NoSQL database such as theMicrosoft® SQL Server, the Oracle® servers, the MySQL® database, etc.The cloud server 216 may be deployed in a cloud environment managed by acloud storage service provider, and the databases may be configured ascloud based databases implemented in the cloud environment.

The cloud server 216 which may include an input-output device usuallycomprises a monitor (display), a keyboard, a mouse and/or touch screen.However, typically there is more than one computer server in use at onetime, so some computers may only incorporate the computer itself, and noscreen and no keyboard. These types of computers are typically stored inserver farms, which are used to realise the cloud network used by thecloud server 216 of some embodiments of the invention. The cloud server216 can be purchased as a separate solution from known vendors such asMicrosoft and Amazon and HP (Hewlett-Packard). The cloud server 216typically runs Unix, Microsoft, iOS, Linux or any other known operatingsystem, and comprises typically a microprocessor, memory, and datastorage means, such as SSD flash or Hard drives. To improve theresponsiveness of the cloud architecture, the data is preferentiallystored, either wholly or partly, on SSD i.e. Flash storage. Thiscomponent is either selected/configured from an existing cloud providersuch as Microsoft or Amazon, or the existing cloud network operator suchas Microsoft or Amazon is configured to store all data to a Flash basedcloud storage operator, such as Pure Storage, EMC, Nimble storage or thelike. Using Flash as the backbone storage for the cloud server 216 ispreferred despite its high cost due to the reduced latency that isrequired and/or preferred for retrieving user data, user preferences,and data related to mobile/software applications etc.

The GPU 206 or 218 may refer to an electronic circuit designed tomanipulate and alter computer graphics, images, and memory to acceleratethe analysis and creation of images/patterns. GPUs can be used inembedded systems, mobile phones, personal computers, workstations, gameconsoles, etc. The GPU 206 or 218 may be selected, for example, from AMDGPUs, Nvidia GPUs, Intel GPUs, Intel GMA, Larrabee, Nvidia PureVideo,SoC, etc. In some embodiments of the invention, typically the machinelearning or Artificial Intelligence parts of the processing areconfigured to be executed by the GPU, due to the large number ofparallel processing or comparative processing required in machinelearning.

The system 200 may be configured as a mobile terminal computer,typically a smartphone and/or a tablet that is used to manage tasks ofthe user by operating software applications installed on the smartphone,etc. The system 200 is typically a mobile smartphone, such as iOS,Android or a Windows Phone smartphone. The GPU 206 or is present in thesmartphone or GPU 218 or 518 present in cloud server 216 or 516respectively, may process data of software applications installed on thesmartphone. The processed data may be used for managing the tasks of theuser. All of the three configurations are possible in accordance withsome embodiments of the invention, the first where the GPU 206 is onlyon the mobile client device 200, the second where the GPU 218 is only onthe cloud server 216, the third where both the cloud server 218 and themobile terminal device 200 have the GPU 206, 218.

However, it is also possible that the system 200 is a mobile station,mobile phone or a computer, such as a PC-computer, AppleMacintosh-computer, PDA-device (Personal Digital Assistant). The system200 could further be a device having an operating system such as any ofthe following: Microsoft Windows, Windows NT, Windows CE, Windows PocketPC, Windows Mobile, GEOS, Palm OS, Meego, Mac OS, iOS, Linux, BlackBerryOS, Google Android and/or Symbian or any other computer or smart phoneoperating system.

The description encompasses several modules, wherein the modules are tobe interpreted as programmed instructions stored in a segment of thememory 208 or 508, which when executed by the processor 204 or GPU 206,218 performs certain functionalities.

Any features of embodiment 20 may be readily combined or permuted withany of the other embodiments 10, 11, 21, 30, 31, 40, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

In phase 102, the data analyzing module 210 of the mobile secretarycloud application or the GPU 206 and/or 218 reads data from anothersoftware application. Another software application indicates a mobileapplication different from the mobile secretary cloud application.Reading the data signifies extracting data entries related to a user.For example, when a calendar application is accessed, all eventsscheduled in a particular month, dates and timings of such events, andpriorities associated with such events are accessed. The events maycomprise meetings, interviews, journeys, birthdays, anniversaries, etc.In one case, the GPU 206 and/or 218 may read the data belonging toanother software application from Random Access Memory (RAM) or cachememory. Further, metadata belonging to the data may also be fetched inone embodiment.

In another embodiment, an email application may be accessed to retrieveall emails present in a mailbox of the user. The data analyzing module210 of the mobile secretary cloud application or the GPU 206 and/or 218may access the email application. All emails present in the mailbox i.e.received, sent, transmission in progress, and recently deleted may beaccessed.

In phase 104, the data modifying module 212 of the mobile secretarycloud application or the GPU 206 and/or 218 operates another softwareapplication based on data read from another software application. Forexample, upon accessing data from a messaging application, the datamodifying module 212 or the GPU 206 and/or 218 may manage messagespresent in the messaging application. In one case, the data modifyingmodule 212 or the GPU 206 and/or 218 may read details related to eachmessage present in the messaging application, such as details ofsenders, content of a message, and contents of other messages receivedfrom a sender. Based on processing of such details, the data modifyingmodule 212 or the GPU 206 and/or 218 may segregate each message into arespective category, such as segregation by category of message,segregation based on sender of messages, and segregation by priority ofmessages.

In another embodiment, upon accessing the emails, the data modifyingmodule 212 or the GPU 206 and/or 218 may manage the emails. The datamodifying module 212 or the GPU 206 and/or 218 may sort the emails basedon priority, time, name of sender(s), name of receiver(s), and contentsof the emails. For example, while the user receives an email towards anacknowledgement of an online order, an instant pop-up message may bedisplayed on the system 200. Successively, based on processing ofcontent present in the email, the data modifying module 212 or the GPU206 and/or 218 may mark the email as read, automatically archive, and/ordelete the email.

In one embodiment, the GPU 206 and/or 218 may utilize ConvolutionalNeural Network (CNN) for processing the data related to the messagingapplication, email application, or other applications. For example,there could be numerous messages present from several senders, and theGPU may perform multi-thread processing for obtaining precise and timelyoutput based on the processing. In one case, the GPU may compriseCombined Unified Device Architecture (CUDA) for enabling themulti-thread processing of the data. For example, the GPU 206, 218and/or processor 204 could generate draft response emails in response toreceived emails. To these emails the GPU 206, 218 and/or processor 204would automatically write text that they have machine learned to berelevant or deduce by Artificial Intelligence that will almost certainlybe present in the response email. For example, if there are multipleemails that address a certain party “Dear Sir/Madam”, the mobilesecretary cloud application will begin the draft response email in thatway too.

In phase 106, the Artificial Intelligence (AI) module 214 of the mobilesecretary cloud application or the GPU 206 and/or 218 utilizesArtificial Intelligence (AI) to operate another software application.The AI module 214 or the GPU 206 and/or 218 operates another softwareapplication for assisting a user and/or to perform a task for the user.In one case, to segregate the messages present in the messagingapplication, the AI module 214 or the GPU 206 and/or 218 utilizes AI forprocessing of different details related to each message. AI may be usedto process content of each message to segregate each message into arespective category. Processing the content of each message may compriseprocessing of title, words, hyperlinks, and dates present in eachmessage. Based on processing of such content, the AI module 214 or theGPU 206 and/or 218 utilizing AI may segregate each message into arelated category.

In addition to analysing the semantic content of messages with AI, themessages can be visually analysed using AI. Consider an example of usingconvolutional neural networks (CNN) for recognizing an email that justarrived in the inbox. CNN refers to artificial neural networks thatmodel visual perception by an animal or a human. The CNN algorithms maybe employed for image recognition tasks preferably emulating the waythat a human secretary views an email in accordance with someembodiments of the invention. The CNN comprises multiple layers ofreceptive fields that are small neuron collections configured to processportions of an input image. The output of each layer is successivelytiled such that the input regions overlap to obtain a representation ofthe original image. In an embodiment, a deep learning framework calledCaffe that uses C++, MATLAB, and Python programming languages is usedfor implementing the CNN. Caffe is a CNN library that is configured tosupport both CPU 290 and GPU operations. In this example, the GPU 210 or240 used by the mobile secretary cloud application 200 may for examplebe a NVIDIA GPU with 15 GB RAM.

A pre-defined dataset comprising, for example, 30000 images oflegitimate business emails and spam emails is used as a training datasetfor training the network. The training dataset comprises labelsassociated with each image. In an example, the training dataset isdownloaded from Kaggle which is a predictive modelling and analyticsplatform. The labelled images 422 and 424 are pre-processed and storedin a Python script format.

In this example, the GPU 206 and/or 218 executes histogram equalizationon the labelled images 422 and 424 of the training dataset. Histogramequalization is a technique used to adjust image intensities by usingthe image's histogram features. Histogram equalization enhances contrastof the images 422 and 424. The images resulting after histogramequalization of the images 422 and 424 are illustrated by images 426 and428, respectively. The GPU 206 and/or 218 performs image resizing toresize the images of the emails, for example, to a 227×227 format. Eachimage of email 430, 432 is labelled after performing the histogramequalization. The training dataset is then divided into 2 subsets. Firstsubset 434 called the training set comprises ⅚^(th) portion of thetraining images that are used for training a model. The second subsetcalled the validation set 436 comprises ⅙^(th) portion of the trainingimages of emails that are used for calculating and validating accuracyof the model. The subsets 434 and 436 are stored in the cloud databaseof the cloud server 216.

Features such as, histogram of oriented gradients (HoG), Scale-invariantfeature transform (SIFT), etc., of the images are extracted from thetraining images 430 and 432 by using a feature extraction software suchas, MATLAB. The extracted image features provide a description offeatures of an object present in an image 430 or 432 that are used inimage classification. Once the subsets 434 and 436 are created, the GPU206/218 generates the mean image for the training data. The GPU 206/218subtracts the mean image from each input image of the training set 434.The GPU 206/218 then performs feature standardization to make eachfeature in the dataset have a zero mean and a standard deviation of 1such that all image data features are normalized. Featurestandardization is used to ensure that measurement comparisons betweenfeatures that may have different units (such as audio signals and pixelvalues of the image data) are normalized. In feature standardization theimage features are centred on a zero mean with a standard deviationof 1. The mean image of the training set 434 is calculated. The meanimage is subtracted from each image in the training set 434. Theresulting value of each image is divided by its standard deviation. Theresulting value of each image feature is hence normalized and can befurther used for creating the training model.

The GPU 206/218 then defines the training model by selecting CNNarchitecture. In this example, the GPU 206/218 uses a CNN architecturemodel such as, Alexnet for defining the model. Alexnet is a CNNframework executed on GPUs implemented in CUDA. CUDA is a parallelcomputing platform and an application programming interface (API) modelcreated by Nvidia that can be used for general purpose processing alongwith graphics processing.

The model is then optimized using a solver algorithm. The solveralgorithm is a type of a stand-alone computer program or a softwarelibrary that is configured to optimize the training model by computingan accuracy of the model using the training dataset. The solveralgorithm computes the accuracy by iteratively using the validation set436. For example, the solver algorithm may use the validation set 436for every 1000 iterations in an optimization process of 40000 iterationsthat takes a snapshot of the trained model at every 5000^(th) iteration.

The GPU 206/218 then performs model training using the results of thesolver algorithm. During the training process, the GPU 210/240 monitorslosses and the model accuracy. In an example, Caffe takes a snapshot ofloss and model accuracy of the trained model at every 5000^(th)iteration. Then the GPU 210/240 plots a learning curve of the losses asa function of the number of iterations as depicted in the graph 438 ofFIG. 2B. Multiple iterations are performed until a steady-state accuracyrate is achieved. For example, as can be seen in the graph 438 thetrained model achieves an accuracy rate of about 90% that stopsimproving after about 3000 iterations.

Once the trained model is ready, the GPU 206/218 starts predictingaccuracy of unseen images from a testing dataset downloaded from aKaggle platform. The GPU 206/218 reads an image of an email from thetesting dataset, processes the image, and calculates a probability ofaccuracy, for example, 0 for non-business email, and 1 for a legitimatebusiness email. For example, if an accuracy rate of 98% is achieved,that image is considered to be of an legitimate business email.

In one embodiment, the model is stored in the cloud database of thecloud server 216 which is accessible by the mobile secretary applicationof the terminal device 200 via the network 220. In another embodiment,the model may be stored in the local memory 208 of the smartphone 200.The GPU 206 and/or 218 analyses an input image of an inbox email 420using the model. The image of the inbox email 420 is compared with thefeatures of a legitimate business email stored in the model. The GPU 206and/or 218 generates an output 245 that identifies the inbox email to bea legitimate business email 245 if the comparison yields a high accuracyrate. In this example, the GPU 206 and/or 218 identifies that the inputimage is of a legitimate business email that is related to a legalmatter. The GPU 206 and/or 218 identifies the business email 245 torelate to a court case the user has. The CPU 204 may compare historicaldata associated with the business email 245 stored in the local memoryof the mobile terminal 200 or the cloud server 216 to determine that theoutput, business email 245 relates to a court case that has the maintrial in 3 days in the calendar application. Hence, the CPU 204 mayrecommend the user to read the email right away. In an embodiment, thebusiness email 245 is placed on the idle screen scale of the mobileterminal 200. A draft response email may be drafted by the mobilesecretary cloud application into the draft folder of the emailapplication, for further editing and approval by the user. Subsequent tothe response, the email 245 is placed into folder “legal” in the emailapplication. The end result of some embodiments of this invention shouldbe the computerized replica of the process where a human secretary looksat the email to see what it is like, and/or reads all or part of it, andthen sorts it to the relevant folder, and prepares obvious furtheraction, for example a draft response to the arrived email.

By “machine learning” or “artificial intelligence” we mean that thecomputer system has been trained to make determinations based on atraining set of samples, and/or has been tested with a validation set tohave a known error rate as explained before.

Messages or emails can be classified either by analysing themsemantically, or visually as explained before. It is also in accordancewith some embodiments of the invention to use the semantic and thevisual AI analyses together to classify the messages or emails even moreaccurately. The visual AI analysis as explained in the above can beapplied mutatus mutandis toother secretarial tasks that would be done bythe visual perception of the human secretary in accordance with someembodiments of the invention. Preferably the GPU carries out the visualAI tasks, and the CPU carries out the semantic AI tasks in someembodiments of the invention.

Any features of embodiment 21 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 30, 31, 40, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

Although the example discussed herein is provided with reference tousing convolutional neural networks that uses the deep learningframework Caffe, C++, MATLAB, and Python programming languages, theNVIDIA GPU, the Kaggle dataset, and the Alexnet CNN architecture model,it is to be understood that the mobile client 200 and the cloud server216, in another embodiment, may be implemented using any other deeplearning algorithm that uses any other framework, programming language,GPU, dataset, and/or architecture model. The example included herein isdescribed with reference to the publication “A PRACTICAL INTRODUCTION TODEEP LEARNING WITH CAFFE AND PYTHON”, which is included here as areference.

In another embodiment, to segregate the emails present in the emailapplication, the AI module 214 or the GPU 206 and/or 218 utilizes AI forprocessing of different details related to each email. AI may be used toprocess content of each email, and to segregate each email into arespective category. Processing the content of each email may compriseprocessing of title, content, hyperlinks, dates, images, and attachmentspresent in each email. Based on processing of such semantic content, theAI module 214 or the GPU 206 and/or 218 and/or CPU utilizing AI maysegregate each email into a related category, so that emails areorganized into folders of different categories, and email classified tobe in the same semantic category are placed in the same folder.

In one embodiment, the GPU may utilize “Caffe” as the AI for operatinganother software application. In some embodiments, processing of data bythe GPU running the AI may provide leverage for faster processing ofdata. For example, the messages may be simultaneously processed based ontheir title, words, hyperlinks, and dates present therein. The semanticAI analysis of some embodiments of the invention ca be possibleimplemented in some embodiments with a ready software package such asNLP, Leiki, Than or similar semantic AI engine.

It is also in accordance with some embodiments of the invention tosegregate emails based on images taken of the emails only, based on theaforementioned content analysis of the emails only, or by using the twomethods in a mix.

Any features of embodiment 10 may be readily combined or permuted withany of the other embodiments 11, 20, 21, 30, 31, 40, 41, 50, 60, 62and/or 63 in accordance with the invention.

FIG. 1B demonstrates an embodiment 11 of a method of implementation of amobile secretary cloud application for operating a third softwareapplication, using another software application, as a flow diagram. Themethod could be implemented in a system identical or similar toembodiment 20 in FIG. 2 and embodiment 50 in FIG. 5 for example. Theend-user of the method could use a user interface identical or similarto that disclosed with embodiment 30, 31, 60, 61, 62, and 63 in FIG. 3A,FIG. 3B, FIG. 6A, FIG. 6B, FIG. 6C, and FIG. 6D.

In phase 108, the data analyzing module 210 of the mobile secretarycloud application using the GPU 206 and/or 218 reads data from anothersoftware application. Another software application indicates a mobileapplication different from the mobile secretary cloud application.Reading the data signifies extracting all entries related to the userfrom another software application. For example, while a calendarapplication is accessed, all events scheduled in a particular month,dates and timings of such events, and priorities associated with suchevents are accessed. The events may comprise meetings, interviews,journeys, birthdays, anniversaries, etc. In one case, the GPU 206 and/or218 may read the data belonging to another software application fromRandom Access Memory (RAM) or cache memory. Further, metadata belongingto the data may also be fetched in one case.

In another case, an email application may be accessed to retrieve allemails present in a mailbox of the user. The data analyzing module 210of the mobile secretary cloud application or the GPU 206 and/or 218 mayaccess the mail application. All mails present in the mailbox i.e.received, sent, transmission in progress, and recently deleted may beaccessed.

In phase 110, the data modifying module 212 of the mobile secretarycloud application or the GPU 206 and/or 218 operates a third softwareapplication based on data read from another software application. Thethird software application indicates a mobile application different fromthe mobile secretary cloud application and another software application.In one case, another software application may refer to the messagingapplication and the third software application may refer to the calendarapplication. The data modifying module 212 and/or the GPU 206, 218and/or processor may extract details related to a scheduled meeting,from at least one message stored in the messaging application. Based onextracted details, such as time and place of meeting/event, the datamodifying module 212 or the GPU 206, 218 and/or processor 204 mayschedule a reminder/event in the calendar application. Thereminder/event may be scheduled based on date and time of the meeting,or persons to be contacted or met in the meeting.

In another embodiment, the data modifying module 212 or the GPU 206, 218and/or processor 204 may extract details of a journey planned withfriends. Details of the journey may comprise a date and a flight numberfor booking seats for the journey. Based on extracted details, the datamodifying module 212 or the GPU 206, 218 and/or processor 204 may bookflight tickets using a related application. For example, the extracteddetails may comprise text mentioning “journey begins on Dec. 19, 2017 at9:30 AM from Delhi, India to New York, United States and returning backon Dec. 28, 2017.” Using such date, time, and place of origin anddestination, the data modifying module 212 or the GPU 206, 218 and/oroperator 204 may reserve flight tickets automatically.

In phase 112, the AI module 214 of the mobile secretary cloudapplication or the GPU 206, 218 and/or processor 204 utilizes ArtificialIntelligence (AI) to operate the third software application. The AImodule 214 or the GPU 206, 218 and/or processor 204 operates the thirdsoftware application for assisting a user and/or to perform a task forthe user. In one case, to identify and successively extract detailsrelated to the scheduled meeting, the AI module 214 or the GPU 206, 218and/or processor 204 utilizes AI for processing of different detailspresent in the at least one message. Processing the content of eachmessage may comprise processing of title, words, hyperlinks, dates andtimes, and passwords present in the at least one message. Based onprocessing of such content, the AI module 214 or the GPU 206, 218 and/orprocessor 204 may schedule the reminder/event in the calendarapplication.

In one case, the AI module 214 or the GPU 206, 218 and/or processor 204utilizes Artificial Intelligence (AI) to process the text “journeybegins on Dec. 19, 2017 at 9:30 AM from Delhi, India to New York, UnitedStates and returning back on Dec. 28, 2017.” Using AI, the AI module 214or the GPU 206, 218 and/or processor 204 may identify the date, time,and place of origin and destination details related to the journey. Thedata modifying module 212 or the GPU 206, 218 and/or processor searchesfor a flight matching the journey details. Successively, upon finding asuitable flight, the AI module 214 or the GPU 206, 218 and/or processorbooks a required number of flight tickets.

Choosing which ticket to buy, or present to user for purchase isdetermined similarly as explained before with classifying emails. Ahuman secretary would look at the emails, and see the typing, layout,language and everything to make a visual determination of whether theemail is a legitimate business email. Choosing a ticket is a slightlydifferent process. A human secretary would not look and determine thechoice by appearance. Instead, price, duration, departure time, arrivaltime and transit time in a stopover are the key factors in choosing whatticket to buy.

First a training set of e.g. 30,000 flight tickets is created. Avalidation set comprising ⅙^(th) of the flights is also created. Thenmobile secretary cloud application is taught what flights are acceptableand what are unacceptable. For example, with the validation set, onlythose fights with a low price, short transit time, short duration,coinciding departure and arrival times are selected. This is done forexample by calculating the normalized standard deviations for eachparameter and assigning them weights. The ticket that deviates the leastfrom an acceptable ticket is either bought or presented to the user forpurchase automatically.

Any features of embodiment 11 may be readily combined or permuted withany of the other embodiments 10, 20, 21, 30, 31, 40, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

FIG. 3A demonstrates an embodiment 30 of software program product userinterfaces 300 and 306 in accordance with the invention as a screen shotdiagram. The user interfaces show the mobile secretary cloud applicationoperating another software application. The user interfaces could bedisplayed for example on a display screen of a mobile client device i.e.a smartphone.

The software program product is stored on a non-transient memory mediumi.e. the memory 208 of the system i.e. mobile client device 200 and thecloud server 216. The software program product may also be distributedbetween the memory 208 of the mobile client device 200 and the cloudserver 216, so that some parts of the software program product reside inthe memory 208 and some parts of the software program product reside onthe cloud server 216.

In an embodiment, a software program product user interface 300 is shownto comprise a mobile secretary cloud application 302 operating anothersoftware application i.e. a messaging application 304. The mobilesecretary cloud application 302 reads data from the messagingapplication 304 and operates the messaging application 304 based on readdata. The read data may include contents of messages, details of sendersof the messages, date and time of receipt of the messages etc. All suchdetails may be processed using AI and accordingly, the messagingapplication may be operated. In one case, a message may be received froma colleague and the message may ask a user to call his colleague. Duringsuch instance, if it is determined that the user is driving a vehicle,the messaging application may automatically answer to the message thatthe user is driving and that the user shall get in touch after a fewminutes. The user driving the vehicle could be determined based onaccelerometer data of the mobile client device of the user or based on adriving mode set by the user on the mobile client device or fromlocation changes of the mobile client device based on GPS or basestation measurements. Similarly, it is also possible to use images ofemails to determine their type and characteristics as explained before.

In an embodiment, a software program product user interface 306 is shownto comprise a mobile secretary cloud application 308 operating anothersoftware application i.e. a calling application 310. The mobilesecretary cloud application 308 may interact with the callingapplication 310 on user's behalf. In one case, the calling application310 may be connected to an Interactive Voice Response System (IVRS) forbooking flight tickets for the user. The mobile secretary cloudapplication 308 may utilize AI for analyzing all instructions receivedvia the IVRS. For example, the user may be required to set a preferredcommunication language over the IVRS by pressing of a soft key. Themobile secretary cloud application 308 may determine English aspreferred communication language of the user, based on previouscommunication of the user over calls, language of content present inmessage, mails, etc. In one case, the mobile secretary cloud applicationmay analyze user's interaction with the IVRS over several calls, tolearn user's preferences and other user data provided by the user, overthe IVRS. The other user data may comprise personal details, paymentdetails, and details for user identity verification. Successively, themobile secretary cloud application 308 may respond to select English asthe preferred language of the user by selecting the soft key assigned toEnglish. Thereafter, the mobile secretary cloud application 308 mayprovide other inputs to the IVRS, such as providing personal details ofthe user, answering to authentication questions for the user, andselecting a particular department to be contacted via the IVRS either byemulating soft key presses, or reading out answers with synthesizedvoice down the phone line. Thus taking the burden of interacting withthe IVRS, or a human service agent, when the answer to the questionasked by the IVRS or the human service agent is known.

In one preferred embodiment, the mobile secretary application gets ahuman service agent on the line. Remember in the old times, whensecretaries would get people “on the line”, i.e. waiting to thetelephone for the CEO? The mobile secretary cloud application can dothat for the normal consumer. The mobile secretary cloud application 308simply navigates through the choices of the IVRS with selections thatdeviate the least from past selections of the user or users that havebeen recorded. Further, the mobile secretary cloud application mayidentify questions asked by the human service agent, such as “What isyour Date of Birth?” or “What is your Customer number?”, andsubsequently voice recognize these questions, and then dictate the dateof birth of the user, or the customer number of the user down the phoneline with synthesized voice.

Similarly to before, a training set of maybe 30,000 IVRS discussions canbe created. The mobile secretary cloud application and cloud network isthen taught with 5000 IVRS validation set discussions, what is asatisfactory conduct of the IVRS discussion. Typically, this is the IVRSdiscussion that deviates the least from the facts available to themobile secretary cloud application, and the mobile secretary cloudapplication will attempt to conduct that IVRS discussion with a computeror a human service agent on behalf of the user via the phone line to asfar as possible. When the mobile secretary cloud application can nolonger determine a satisfactory response, it may consult the user ordirect that part of the interaction to the human user so that he mayconduct that part of the interaction himself. The mobile secretary cloudapplication may also contain goal based standard call and discussionprotocols, for example “get window seats for flight”. When selected, themobile secretary cloud application calls the airline with the bookingreference obtained from the emails, and conducts the discussion withsoft keys or synthesized voice to obtain a window seat for the user. Ifthat does not work the i mobile secretary cloud application mayautomatically try to obtain the window seat via airline webpages byfilling online forms of the airline or an online travel agent, andoptionally purchasing tickets thereby with the user's credit carddetails stored in the cloud network or the mobile terminal. Of coursethe order in which the call and the website are used could be reversed:the mobile secretary cloud application could try first to obtain thewindow seat via the webpage, but if that does not work it could call theairline as explained before to obtain a window seat. For activities likethis separate training sets and validations sets could be configured,for example by recording 30,000 phone conversations where a window seatis obtained, or by recording 30,000 web sessions where a window seat isobtained.

Any features of embodiment 30 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 31, 40, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

FIG. 3B demonstrates an embodiment 31 of a software program product userinterface 312 in accordance with the invention as a screen shot diagram.The user interface 312 shows the mobile secretary cloud application 314utilizing another software application for operating a third softwareapplication. The user interface could be displayed for example on adisplay screen of a mobile client device i.e. a smartphone.

In an embodiment, a software program product user interface 312 is shownto comprise a mobile secretary cloud application 314 utilizing anothersoftware application for operating the third software application, whereanother software application and the third software application isselected from a messaging application 316, a calling application 318,and a map application 320. In one embodiment, the mobile secretary cloudapplication 314 accesses data present in the messaging application 316for operating the calling application 318. For example, the messagingapplication may comprise a message including calling details of an AirConditioning (A.C.) repairing agency. Upon retrieving such details fromthe message, the mobile secretary cloud application 314 mayautomatically call the A.C. repairing agency and may answer to an IVRSsystem over the call, for arranging repair of the A.C. In another case,the messaging application may comprise another message including detailsof a venue for attending a meeting by the user. Based either on a user'srequest or automatically, on the date of the meeting, the mobilesecretary cloud application 314 may operate the map application 320 forsetting a navigation path, to guide the user for reaching the venue ofmeeting. In some embodiments this navigation path is sent to aself-driving car configured to transport the user.

Any features of embodiment 31 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 30, 40, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

FIG. 4A illustrates an embodiment 40 of a flow chart showing a method ofimplementation of a mobile secretary cloud application for operatingseveral other software applications. The different phases/stepsmentioned in FIG. 4A are not necessarily performed in mentionedsequence, but could be performed in different sequences orindependently.

FIG. 5 illustrates an embodiment 50 of a block diagram of a system 500executing the mobile secretary cloud application. The system comprisesinterface(s) 502, processor 504, Graphical Processing Unit (GPU) 506,and memory 508. The memory 508 comprises a mail and message processingmodule 510, a call management module 512, and a navigation module 514.In an embodiment, the system 500 is integrated with a cloud server 516,via a communication network 520. The cloud server 516, the GPU 518, andthe communication network 520 are similar to the cloud server 216, GPU218, and the communication network 220 of FIG. 2 respectively. Differentphases of FIG. 4A are now explained in conjunction with modules of FIG.5 .

Any features of embodiment 50 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 40, 41, 60, 61, 62,and/or 63 in accordance with the invention.

In phase 402, the mobile secretary cloud application replicatessecretary functions i.e. the mobile secretary cloud application performstasks for the user and/or provides assistance to the user in managingtasks by operating other software application(s) installed on the mobileclient device like a human secretary would.

In phase 404, another software application is an email or messagingapplication 604. The email and message processing module 510 of themobile secretary cloud application 602 or the GPU 506, 518 and/orprocessor 504 analyses messages and emails, and automatically sorts themessages and emails into different folders. Analysis of the messages andemails may be performed by utilizing AI, that processes contents of themessages and emails and other information related to the messages andemails, for sorting the messages and emails into different folders 606a-606 c. The artificial intelligence can be realized by teaching themobile secretary cloud application to distinguish an email from theimage of the email, with training sets and validations sets of images ofemails for example as explained before. Alternatively or additionally,the AI can classify an email based on metadata and semantic content, inwhich case a training set of emails and validations set of emails wouldbe used so, that the artificial intelligence secretary application istaught to classify emails based on their semantic content, and metadata,such as time, recipient, sender and the like. Further these two AImethods can be used together in a mix.

In phase 406, the email and message processing module 510 or the GPU506, 518 and/or processor 504 analyses messages and selects anindication sound (amongst 616 a-616 c) automatically based onclassification, contents of the messages and sender and/or recipients ofthe messages. The email and message processing module 510 or the GPU506, 518 and/or processor 504 may utilize AI to process contents of themessages and may select an indication sound based on the contents. Forexample, the mobile client device of the user may receive a pleasantmessage, stating “You did an incredible job.” Similarly, the user mayreceive other pleasant messages, such as “Thank you for your kindsupport,” “I am glad to hear about your promotion,” and “love you myson.” The email and message processing module 510 or the GPU 506, 518and/or processor 504, utilizing AI, may learn upon analyzingsemantically several such messages, and may determine that such messagesconvey pleasant gestures.

The email and message processing module 510 or the GPU 506, 518 and/orprocessor 504 may be taught to learn or know the semantic meanings ofwords present in the emails and messages. For example, in oneembodiment, the words “incredible,” “thank you,” “kind,” “glad,”“promotion,” and “love” infer pleasant gestures from a sender, and theemail and message processing module 510 or the GPU 506, 518 and/orprocessor 504 may associate a positive meaning to such words. Based onsuch learning, the email and message processing module 510 or the GPU506, 518 and/or processor 504 may play uplifting and positive audio asthe indication sound.

In a contrasting case, the mobile client device of the user may receivea harsh message, stating “poor job.” Similarly, the user may receiveother harsh messages, such as “you've passed the deadline,” “I won'ttolerate such careless attitude,” and “hate you.” The email and messageprocessing module 510 or the GPU 506, 518 and/or processor 504 utilizingAI, may learn upon several such messages, and may determine that suchmessages convey harsh and negative gestures. The email and messageprocessing module 510 or the GPU 506, 518 and/or processor 504 may betaught to attribute a negative meaning to these words present in theemails and messages. For example, in the present embodiment, the words“poor,” “passed deadline,” “won't tolerate,” and “hate” infer harsh andnegative gestures from a sender, and the email and message processingmodule 510 using the GPU 506, 518 and/or processor may be taught to knowor learn such semantic meaning. Based on such learning or knowledge, theemail and message processing module 510 or the GPU 506, 518 and/orprocessor 504 may play a loud beep or warning signal as the indicationsound.

Further, the email and message processing module 510 or the GPU 506, 518and/or processor 504 may also derive meanings based on analysis ofsmileys/emoticons and special characters present in the emails andmessages. The special characters may include “!!,” “?,” and other knownspecial characters. For example, the user may receive a message stating“you again!!” or “what have you done?.” The email and message processingmodule 510 or the GPU 506, 518 and/or processor 504 may learn and/ordistinguish the delivery of a harsh gesture through such message. Basedon such learning, the email and message processing module 510 or the GPU506, 518 and/or processor 504 may play a warning sound or a loud beep asthe indication sound.

In phase 408, another software application is a telephony application626. Using the telephony application 626, the call management module 512of the mobile secretary cloud application 622 or the GPU 506 and/or 518may make phone calls on behalf of the user, may automatically answer tomessages or compose draft replies, and may answer to incoming calls onbehalf of the user, and may prompt queries and answer to queries onbehalf of the user, using a synthesized voice on a phone line. Forexample, during an IVRS phone call the user's phone number and customernumber may be asked. The call management module 512 or the GPU 506, 518and/or processor 504 may recognize the asked question by voicerecognition software and provide answers to the recognized questionsbased on previous calls or data available in the emails messages, or thefilesystem, and may respond to the IVRS or customer attendant in thecall center by reading the information using synthesized voice.

In another embodiment, the user may need to book flight tickets fromMumbai, India to New York, United States and details of such journey maybe stored in an email or message. Based on approval of the user orautomatically, the mobile secretary cloud application 622 accesses suchdetails for booking the flight tickets for the user. The call managementmodule 512 or the GPU 506 and/or 518 may call a customer care number ofan airline, use a voice synthesizer to read out and voice recognitionsoftware to recognize details for communicating over the call, and thusbook the flight tickets over the phone.

In phase 410, another software application is a map application 634. Thenavigation module 514 of the mobile secretary cloud application 632using the GPU 506, 518 and/or processor 504 using the map application634 provides a route calculator that provides a combined route ofdriving and walking based on emails, calendar entries, and locationsearch. The navigation module 514 or the GPU 506, 518 and/or processor504 upon accessing data present in the emails and the calendar entriesmay determine a date and time of reaching a location by the user. Thenavigation module 514 or the GPU 506, 518 and/or processor 504 maydetermine multiple routes for reaching the location. Further, thenavigation module 514 or the GPU 506, 518 and/or processor 504 mayaccess recent location searches performed by the user, and may utilizesuch information for filtering the multiple routes. In one case, thenavigation module 514 or the GPU 506, 518 and/or processor 504 mayprovide the user with a route which is most familiar to the user, i.e.the user has been through at least once.

The method explained with Graph 438 in FIGS. 2A and 2B and elsewhereearlier in the description can be used with all third applications email510, calls 512, navigation 514. The mobile secretary cloud applicationand cloud server 516 is first provided with a large training set,spanning the diversity of emails, phone conversations, or navigationroutes. The mobile secretary cloud application is then taught with avalidation set of emails, phone conversations or navigation routes whatis an acceptable choice for a user. Once the error rate is diminishedbelow a threshold with a large number of iterations as shown in graph438 of FIG. 2B, the model is deployed to users. Naturally machinelearning files can be continuously maintained, updated and improved onthe cloud server network 516 from which the machine learning files andtheir updates are made available to the terminal devices 200, 500.

Any features of embodiment 40 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 41, 50, 60, 61, 62,and/or 63 in accordance with the invention.

FIG. 4B illustrates a flow chart showing a method of implementation of amobile secretary cloud application for operating several third softwareapplications, using the mobile secretary cloud software application,according to some embodiments of the invention. The differentphases/steps mentioned in FIG. 4B are not necessarily performed inmentioned sequence, but could be performed in different sequences orindependently. Different phases of FIG. 4B are now explained inconjunction with modules of FIG. 5 .

In phase 412, the mobile secretary cloud application replicatessecretary functions i.e. the mobile secretary cloud application performstasks for the user and/or provides assistance to the user in managingtasks by operating software application(s) installed on the mobileclient device like a professional secretary would.

In phase 414, the third software application operated by the mobilesecretary cloud application 602 is an email or messaging application604. The email and message processing module 510 or the GPU 506, 518and/or processor 504 of the mobile secretary cloud application 602analyses messages and automatically sorts the messages to differentfolders 606 a-606 c shown in FIG. 6A. Along with information present inthe messages and emails, the email and message processing module 510 orthe GPU 506, 518 and/or processor 504 may utilize data of anotherapplication, for example a calendar application, for sorting themessages and emails to different folders 606 a-606 c. In one embodimentthe folders 606 a-606 c have different urgency levels, like one day, oneweek, one month and so on. The mobile secretary cloud application movesthe emails to the different folders 606 a-606 c and in between thefolders 606 a-606 c as the emails become more or less urgent, as newentries are marked into the calendar, and so forth.

In phase 416, the email and message processing module 510 or the GPU506, 518 and/or processor 504 analyses messages and selects anindication sound (amongst 616 a-616 c) automatically based on contentsof the messages and sender and/or recipients of the messages. In onecase, the email and message processing module 510 or the GPU 506, 518and/or processor 504 may utilize AI to process contents of the messagesand may select an indication sound based on the contents, or the folderto which the message is classified.

In phase 418, the third software application is a telephony application626. Using the telephony application 626, the call management module 512or the GPU 506, 518 and/or processor 504 of the mobile secretary cloudapplication 622 makes phone calls on behalf of the user, mayautomatically answer to messages or incoming phone calls, and may answerto queries on behalf of the user, using a synthesized voice on the phoneline. For example, during an IVRS phone call the user's phone number andcustomer number may be asked. The call management module 512 mayrecognize the question by voice recognition software and find requestedinformation in a digital wallet of the user or other application storingsuch detail, and may respond on the IVRS by reading the informationusing the synthesized voice. In another embodiment, the user may need tobook flight tickets from Mumbai to New York and details of such journeymay be stored in an email or message. Based on approval of the user, thecall management module 512 or the GPU 506, 518 and/or processor 504access such details for booking the flight tickets for the user. Thecall management module 512 or the GPU 506, 518 and/or processor 504 forexample calls a customer care number of an airline, uses a voicesynthesizer to read such details for communicating over the call, andthus books the flight tickets. Alternatively, the mobile secretary cloudapplication may be configured to operate an e-commerce application tobook the flights. The acceptable flight or acceptable flights forpurchase can be determined as explained before, by providing a trainingset and validation set of flights and teaching the mobile secretarycloud application to select an acceptable flight that, for exampledeviates from the model acceptable flight the least in a number ofparameters such as duration, price, departure and arrival time and/orstopover duration.

In phase 420, the third software application is a map application 634.The navigation module 514 of the mobile secretary cloud application orthe GPU 506, 518 and/or processor 504 using the map application providesa route calculator that provides a combined route of driving and walkingbased on emails, calendar entries, and location search performed by theuser for conducting the activities of the day at various locations andtimes. The navigation module 514 or the GPU 506, 518 and/or processor504 upon accessing data present in the emails 636 and the calendarentries 638 may determine a date and time of reaching a location by theuser. The mobile secretary cloud application may determine multipleroutes for reaching the location. Further, the navigation module 514 orthe GPU 506, 518 and/or processor 504 may access recent locationsearches 640 performed by the user, and may utilize such information forfiltering the multiple routes. In one case, the navigation module 514 orthe GPU 506, 518 and/or processor may provide the user with a routewhich is most familiar to the user, or which route when followed allowsthe user to complete the maximum number of errands or tasks on his to dolist.

The aforementioned secretarial process of optimizing routes, agendasand/or timetables naturally results in the knowledge of events that theuser has actually visited. The location and time of the mobile stationcan be followed and calendar entries and Internet search can beharnessed along with call application data to construct a history ofevents that the user has attended. Many of these events are likely to berecurring: conferences happen every year, the barber has to be visitedonce per month, the dentist once per year, the accountant needs to becontacted every 3 months to prepare a quarterly report and so forth. Themobile secretary cloud application can automate the rebooking of theuser into these events. In the first stage, the mobile secretary cloudapplication may populate the calendar with entries when the event takesplace or should take place. The mobile secretary cloud application maysearch the Internet to determine the 2019 dates of a conference that theuser attended in 2018, and mark them up in the calendar. The mobilesecretary cloud application may also register the user to thisconference automatically or upon approval of the user. The mobilesecretary cloud application may book barber and dentist appointments ata convenient time and place automatically or upon approval of the useras explained before via phone or the webpage of the barber or dentist.All of these activities can be done without or almost without any userparticipation by the mobile secretary cloud application merely repeatingrecorded telephone conversations or webpage sessions and adapting themwith AI to the current or upcoming situation. If plans change the mobilesecretary cloud application may automatically cancel activities orevents on behalf of the user, by similarly executing web sessions orphone conversations of cancellation on behalf of the user, adapted withAI to the current situation.

Any features of embodiment 41 may be readily combined or permuted withany of the other embodiments 10, 20, 21, 30, 31, 40, 50, 60, 61, 62,and/or 63 in accordance with the invention.

FIG. 6A demonstrates an embodiment 60 of a software program product userinterface 600 in accordance with the invention as a screen shot diagram.The user interface 600 shows the mobile secretary cloud application 602operating an email/messaging application 604. The user interface 600could be displayed for example on a display screen of the mobile clientdevice i.e. a smartphone 500.

In an embodiment, the software program product user interface 600 isshown to comprise the mobile secretary cloud application 602 operatingthe email/messaging application 604. The mobile secretary cloudapplication 602 reads data from the email/messaging application 604 andoperates the email/messaging application 604 based on read data. Theread data may include contents of messages, details of senders of themessages, date and time of receipt of the messages etc. All such detailsmay be processed using AI and accordingly, the messaging application maybe operated. In one case, based on processing of the read data, themobile secretary cloud application 602 may store official work relatedmails and messages into folder 606 a, emails and messages from friendsand families into folder 606 b, and other emails and messages, such asfrom banks, telecom operators, and other service providers into folder606 c. For example, the messages and emails may be sorted into differentfolders based on the senders, month of receipt, and details of themessages/emails. The mobile secretary cloud application may also takeimages or photographs of the emails to graphically process the emails asexplained before in association with FIGS. 2A and 2B. Similarly, thephotographic/visual classification of emails or messages may be donetogether with the semantic classification so that these two methods areused in a mix. It is also in accordance with some embodiments of theinvention for the mobile cloud secretary application to search theInternet to gain further information about sender, recipient, or thelike to assist in the message and/or email classification.

Any features of embodiment 60 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 40, 41, 50, 61, 62,and/or 63 in accordance with the invention.

FIG. 6B demonstrates an embodiment 61 of a software program product userinterface 610 in accordance with the invention as a screen shot diagram.The user interface 610 shows the mobile secretary cloud application 612operating an email/messaging application 614. The user interface 610could be displayed for example on the display screen of the mobileclient device i.e. a smartphone.

In an embodiment, the software program product user interface 610 isshown to comprise the mobile secretary cloud application 612 operatingthe email/messaging application 614. The mobile secretary cloudapplication 612 analyses messages and selects an indication soundautomatically based on contents of the messages and sender and/orrecipients of the messages. In one case, the email and messageprocessing module 510 or the GPU 506, 518 and/or processor 504 utilizesAI to process contents of the messages and selects an indication soundbased on the contents. For example, the mobile client device of the usermay receive a pleasant message, stating “You did an incredible job.”Similarly, the user may receive other pleasant messages, such as “Thankyou for your kind support,” “I am glad to hear about your promotion,”and “love you my son.” The email and message processing module 510 orthe GPU 506, 518 and/or processor 504, utilizing AI, may be taught thatthe semantic meaning of such messages convey pleasant gestures. Theemail and message processing module 510 or the GPU 506, 518 and/orprocessor 504 may also continuously learn and update machine learningfiles, or deep learning files on the terminal or in the cloud, based onprocessing of words present in the emails and messages. Typically insome embodiments these machine learning files, or deep learning files orArtificial Intelligence files contain training set and/or validation setdata. For example, in present embodiment, the words “incredible,” “thankyou,” “kind,” “glad,” “promotion,” and “love” infer pleasant gesturesfrom a sender, and the email and message processing module 510 or theGPU 506, 518 and/or processor may learn such meaning, and/or attributesuch semantic meaning to these words. Based on such learned orattributed meaning, the email and message processing module 510 or theGPU 506, 518 and/or processor 504 may play a positive and upliftingaudio 616 a with a loudspeaker as the indication sound.

It is in accordance with some embodiments of the invention to have theArtificial intelligence files used to train and validate the ArtificialIntelligence process on the terminal, in the cloud and/or distributedbetween both of them. Artificial intelligence files, deep learningfiles, machine learning files and self-learning files are usedinterchangeably, and refer to the files used to establish the trainingand validation process producing the artificial intelligence asexplained before for example with FIG. 2B.

In a contrasting case, the mobile client device of the user may receivea harsh message, stating “poor job.” Similarly, the user may receiveother harsh messages, such as “you've passed the deadline,” “I won'ttolerate such careless attitude,” and “hate you.” The email and messageprocessing module 510 or the GPU 506 and/or 518, utilizing AI, may learnupon several such messages, and may determine that such messages conveydissatisfaction from the sender. The email and message processing module510 or the GPU 506, 518 and/or processor 504 may also continuously learnbased on processing of words present in the emails and messages, andupdate the machine learning files on the terminal device and/or thecloud server. For example, in present case, the words “poor,” “passeddeadline,” “won't tolerate,” and “hate” infer dissatisfaction from asender, and the email and message processing module 510 or the GPU 506,518 and/or processor may know, or learn such meaning. Based on suchlearning or knowing the semantic meaning, the email and messageprocessing module 510 using the GPU 506, 518 and/or processor 504 mayplay a loud beep 616 b or warning sound as the indication sound with aloudspeaker.

Further, the email and message processing module 510 or the GPU 506, 518and/or processor 504 may also derive meanings based on analysis ofsmileys/emoticons and special characters present in the emails andmessages. The special characters may include

“!!,” “?,” and other known special characters or emoticons. For example,the user may receive a message stating “you again!!” or “what have youdone?.” The email and message processing module 510 or the GPU 506, 518and/or processor may learn delivery of harsh gesture through suchmessage, or a positive gesture if the emoticon has a positive semanticmeaning. Based on such knowing or learning, the email and messageprocessing module 510 or the GPU 506, 518 and/or processor 504 may playa positive uplifting signal for positive emoticons and a negative audiosignal such as a loud beep 616 b as the indication sound for negativeemoticons. Similarly, other indication sounds 616 c may be used fordifferent special characters or emoticons indicating other expressionsand/or gestures of the emails and messages received by the user.

Any features of embodiment 61 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 40, 41, 50, 60, 62,and/or 63 in accordance with the invention.

FIG. 6C demonstrates an embodiment 62 of a software program product userinterface 620 in accordance with the invention as a screen shot diagram.The user interface 620 shows the mobile secretary cloud application 622operating the calling application 626. The user interface 620 could bedisplayed for example on the display screen of the mobile client devicei.e. a smartphone.

In an embodiment, the software program product user interface 620 isshown to comprise the mobile secretary cloud application 622 operating acalling application 626. The mobile secretary cloud application 622 mayinteract with the calling application 626 on user's behalf. In one case,the calling application 626 may be connected to an Interactive VoiceResponse System (IVRS) for bookings flight tickets for the user. Themobile secretary cloud application 622 may utilize AI for analyzing allinstructions or voice prompts received via the phone line from the IVRS.For example, the user may be required to set a preferred communicationlanguage over the IVRS by pressing of a soft key or number key. Themobile secretary cloud application 622 may determine English aspreferred communication language of the user, based on user data anduser preferences 624 of the user. Successively, the mobile secretarycloud application 622 may respond to select English as preferredlanguage of the user. Thereafter, the mobile secretary cloud application622 may provide other inputs to the IVRS, such as providing personaldetails of the user, answering to authentication questions for the user,and selecting a particular department to be contacted via the IVRS. Themobile secretary cloud application 622 may retrieve all requiredinformation from the user data and user preferences 624 of the user. Forexample, the user data may comprise travelling details, personal detailsand payment details of the user. Thus, the mobile secretary cloudapplication 622 may provide the travelling details over the IVRS and mayfinally provide the payment details, over the IVRS, for booking a flightticket for the user.

Any features of embodiment 62 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 40, 41, 50, 60, 61,and/or 63 in accordance with the invention.

FIG. 6D demonstrates an embodiment 63 of a software program product userinterface 630 in accordance with the invention as a screen shot diagram.The user interface 630 shows the mobile secretary cloud application 632operating the map application 634. The user interface 630 could bedisplayed for example on the display screen of the mobile client devicei.e. a smartphone.

In an embodiment, the software program product user interface 630 isshown to comprise the mobile secretary cloud application 632 operatingthe map application 634. The mobile secretary cloud application 632using the map application 634 provides a route calculator that providesa combined route of driving and walking based on emails, calendarentries, and location search. The emails may be accessed from theemail/messaging application 636, calendar entries may be accessed fromthe calendar application 638, and location search performed by the usermay be accessed from a location search application 640 or Internetbrowser. The mobile secretary cloud application 632 upon accessing datapresent in the emails and the calendar entries may determine a date andtime of reaching a location by the user. The mobile secretary cloudapplication 632 may determine multiple routes for reaching the location.Further, mobile secretary cloud application 632 may access recentlocation searches performed by the user, and may utilize suchinformation for filtering the multiple routes. In one case, the mobilesecretary cloud application 632 may provide the user with a route whichis most familiar to the user, i.e. the user has been through at leastonce.

In one case, the mobile secretary cloud application 632 may identifymultiple calendar entries present in the calendar application 638, for aparticular day. The user may need to visit multiple locations forattending different meetings scheduled on the day. In such case, themobile secretary cloud application 632, using the map application 634may provide a route map to the user, such that a least distance or timewill need to be travelled to visit all the locations.

Any features of embodiment 63 may be readily combined or permuted withany of the other embodiments 10, 11, 20, 21, 30, 31, 40, 41, 50, 60, 61and/or 62 in accordance with the invention.

In the preceding description it has been described how the mobilesecretary cloud application arrives at good or optimum outcomes insecretarial tasks using machine learning and artificial intelligence. Asexplained before, this is typically achieved by taking a training set ofsubstantial size, typically 30,000 samples or so, and teaching thecomputer with a validation set of about 5000 samples what is anacceptable choice and what is not an acceptable choice. This principleis applied to different secretarial tasks regardless of the data formatof the output. For example, if the secretarial task is looking at emailsand judging which emails are genuine and important, the sample sets usedwill be semantic data. i.e. words and content of the email, its metadata(sender, recipient, time, title) and also optionally the image of theemail. The type setting, colors, layout, all visual factors contributeto the decision of how to characterize the email.

If the secretarial task is interaction on the phone, the training setsand validation sets will likely be audio soundtrack and dial toneentries. If the secretarial task is route planning, the training andvalidation sets will be the routes. If the secretarial task is bookingflight tickets, the training and validations sets will be flighttickets.

Typically, the mobile secretary cloud application 602, 612, 622, 632,will be conducting Internet searches from time to time automatically tofind a result(s) that would be satisfactory to the AI model in theterminal device and cloud network. This means that if a good enoughroute, flights, or the like is not found, the secretary application cancontinue to conduct Internet searches to find a satisfactory result forthe user on his behalf.

In the aforementioned description, the mobile secretary cloudapplication 602, 612, 622, 632 has been described, and how the mobilesecretary cloud application arrives at good or optimum outcomes insecretarial tasks using machine learning and artificial intelligence. Itis also in accordance with some embodiments of the invention to combineoptimized results of individual secretarial outcomes to arrive at anoptimized agenda for an entire day for the user. This is achieved asfollows. The terminal computer 200, 500 and cloud network 216, 516 ispresented with a training set of 30,000 or so daily agendas, and avalidation set of a few thousand agendas to teach the mobile secretarycloud application 602, 612, 622, 632 and the cloud server network 216,516 to distinguish between a good daily agenda for the user, and anunacceptable agenda for the user. When the mobile secretary cloudapplication determines the individual outcomes to the secretarial tasks,it will then fit these tasks to a daily agenda. This daily agenda willthen be compared with the AI model, and the agenda that is accepted bythe trained AI model is presented to the user with the secretarialoutcomes in accordance with the agenda. In some embodiments an agendafor a week, month and/or year, or any unit of time is similarlycomposed, using an AI model as explained before mutatis mutandis.

For each secretarial task the mobile secretary cloud application can betrained with deep-learning datafiles. These deep learning data files mayreside on a cloud server to which the client device, i.e. typically amobile phone is connected to via Wi-Fi or cellular connection. Themobile secretary cloud application can be trained to conduct newsecretarial tasks on behalf of the user by providing new sets of deeplearning data files. Similarly secretarial activities that the mobilesecretary cloud application already performs can be improved by updatingdeep learning data files. The training and validation datasets for anyactivity can be updated from time to time so that the secretarialapplication recognizes the need for an appropriate secretarial task withthe best accuracy, and that the secretarial application fulfills thetask with the best accuracy.

The mobile secretary cloud application may record tasks that it performseither in the form of a computing session, a recording of a synthesizedphone conversation, and/or text, and/or images of messages or sessionsthat the mobile secretary cloud application performs. The recordings ofpast tasks may be used to enrich the deep learning data files. The deeplearning data files may reside on either the client device such as themobile phone or in the cloud in accordance with some embodiments of theinvention. The deep learning data files may be shared by one or morecloud servers between different users in some embodiments. In someembodiments the origin of the deep learning data files used to train themobile secretary cloud application is concealed from the user, in orderto protect the privacy of those users from whom the deep learning datafiles originate. It is also in accordance with some embodiments of theinvention that every user generates all or some of their own deeplearning data files.

Embodiments of the invention have been explained in the aforementionedand sizeable advantages of some embodiments of the invention have beendemonstrated. Some embodiments of the invention allow the mobilesecretary cloud application to replicate the secretarial functions to anormal consumer at a very small cost. The mobile secretary cloudapplication produces draft responses to emails and messages, interactson the phone on behalf of the user, and makes purchases such as flighttickets on behalf of the user. The inventive secretarial applicationalso sorts and archives emails and messages to different folders,arranges tasks for the user based on priority and urgency, and selectsroutes and agendas for the day. Similarly, the secretary application602, 612, 622, 632 updates data from one application to the other, forexample emailed agreements of meetings to the calendar and providesdifferent indication sounds alerting the user to the category or vibe ofan incoming message. Any secretarial task can be performed by theintelligent mobile secretary application 602, 612, 622, 632 and cloudnetwork 216, 516 as long as the system is trained to produce anddistinguish an acceptable secretarial work product with training andvalidations sets of the said secretarial work product in accordance withsome embodiments of the invention.

The invention has been explained above with reference to theaforementioned embodiments. However, it is clear that the invention innot only restricted to these embodiments, but comprises all possibleembodiments within the spirit and scope of the inventive thought and thefollowing patent claims.

I claim:
 1. A non-transitory computer readable medium includinginstructions for performing operations for one or more software programproducts to operate with a client device connected to a cloud servernetwork via a communication network, the operations comprising: reading,by at least one of the software program products, data from a messagingapplication; operating, by at least one of the software programproducts, the messaging application, based on the data from themessaging application, wherein operating the messaging applicationincludes analyzing and sorting messages; and using, by at least one ofthe software program products, one or more artificial intelligencemodels trained using data collected from a plurality of users, to atleast one of: (i) operate one or more applications for assisting a userto perform tasks or (ii) use the one or more applications independentlyof the user for performing the tasks on behalf of the user, wherein thetasks include all of: providing one or more draft answers to queriesincluded in inbound messages, automatically to the user via theapplications, comprising: analyzing the inbound messages, using at leastone of the artificial intelligence models, based on content of theinbound messages; and organizing an agenda for the user, using at leastone of the artificial intelligence models, comprising: scheduling a unitof time and a place for one or more activities in the agenda based onavailable time in a user calendar, wherein the cloud server networkrepeatedly updates artificial intelligence files for a number of tasks,and makes the artificial intelligence files available to the softwareprogram products to operate with the client device.
 2. Thenon-transitory computer readable medium of claim 1, wherein at least oneof: (i) providing the one or more draft answers to queries or (ii)organizing the agenda, is further based on a prior input by the user forat least one of the one or more applications.
 3. The non-transitorycomputer readable medium of claim 2, wherein the at least one of: (i)providing the one or more draft answers to queries or (ii) organizingthe agenda, at least partially replicates the prior input by the userusing at least one of the artificial intelligence models.
 4. Thenon-transitory computer readable medium of claim 1, wherein at least oneof: (i) providing the one or more draft answers to queries or (ii)organizing the agenda, is further based on using crowd sourced data withthe at least one of the artificial intelligence models.
 5. Thenon-transitory computer readable medium of claim 1, wherein at least oneof the software program products analyzes messages from the messagingapplication and selects an indication sound automatically based on oneor more of: contents of the messages, sender of the messages or one ormore recipients of the messages.
 6. The non-transitory computer readablemedium of claim 1, wherein at least one of the software program productsprovides a route calculator via a map application that provides acombined route of driving and walking based on data from all of: (i) theemail or calendar application, (ii) a search application, and (iii) anInternet browser.
 7. The non-transitory computer readable memory ofclaim 1, wherein the one or more applications are operated with usingthe one or more artificial intelligence models to produce the agenda. 8.The non-transitory computer readable medium of claim 1, wherein theagenda comprises a plurality of activities and organizing the agenda forthe user comprises determining an optimum total fit of all of thecombined plurality of activities in the agenda for a unit of time. 9.The non-transitory computer readable medium of claim 1, whereinproviding the one or more draft answers to queries included in inboundmessages comprises individually determining draft responses to thequeries, and guiding the user through the responses.
 10. Thenon-transitory computer readable medium of claim 1, wherein the inboundmessages comprises an email, and wherein analyzing the inbound messagescomprises associating a meaning to the email based on semantic contentin the email.
 11. A system for performing operations for one or moresoftware program products to operate with a client device connected to acloud server network via a communication network, comprising: aprocessor; and a memory connected to the processor and storinginstructions for the one or more software program products, wherein theprocessor is configured to execute the software program products toperform the operations, comprising: reading, by at least one of thesoftware program products, data from a messaging application; operating,by at least one of the software program products, the messagingapplication, based on the data from the messaging application, whereinoperating the messaging application includes analyzing and sortingmessages; and using, by at least one of the software program products,one or more artificial intelligence models trained using data collectedfrom a plurality of users, to at least one of: (i) operate one or moreapplications for assisting a user to perform tasks or (ii) use the oneor more applications independently of the user for performing the taskson behalf of the user, wherein the tasks include all of: providing oneor more draft answers to queries included in inbound messages,automatically to the user via the applications, comprising: analyzingthe inbound messages, using at least one of the artificial intelligencemodels, based on content of the inbound messages; and organizing anagenda for the user, using at least one of the artificial intelligencemodels, comprising: scheduling a unit of time and a place for one ormore activities in the agenda based on available time in a usercalendar, wherein the cloud server network repeatedly updates artificialintelligence files for a number of tasks, and makes the artificialintelligence files available to the software program products to operatewith the client device.
 12. The system of claim 11, wherein at least oneof: (i) providing the one or more draft answers to queries or (ii)organizing the agenda, is further based on a prior input by the user forat least one of the one or more applications.
 13. The system of claim12, wherein the at least one of: (i) providing the one or more draftanswers to queries or (ii) organizing the agenda, at least partiallyreplicates the prior input by the user using at least one of theartificial intelligence models.
 14. The system of claim 11, wherein atleast one of: (i) providing the one or more draft answers to queries or(ii) organizing the agenda, is further based on using crowd sourced datawith the at least one of the artificial intelligence models.
 15. Thesystem of claim 11, wherein at least one of the software programproducts analyzes messages from the messaging application and selects anindication sound automatically based on one or more of: contents of themessages, sender of the messages or one or more recipients of themessages.
 16. The system of claim 11, wherein at least one of thesoftware program products provides a route calculator via a mapapplication that provides a combined route of driving and walking basedon data from all of: (i) the email or calendar application, (ii) asearch application, and (iii) an Internet browser.
 17. The system ofclaim 11, wherein the one or more applications are operated with usingthe one or more artificial intelligence models to produce the agenda.18. The system of claim 11, wherein the agenda comprises a plurality ofactivities and organizing the agenda for the user comprises determiningan optimum total fit of all of the combined plurality of activities inthe agenda for a unit of time.
 19. The system of claim 11, whereinproviding the one or more draft answers to queries included in inboundmessages comprises individually determining draft responses to thequeries, and guiding the user through the responses.
 20. The system ofclaim 11, wherein the inbound messages comprises an email, and whereinanalyzing the inbound messages comprises associating a meaning to theemail based on semantic content in the email.
 21. A personal computingdevice connected to at least one cloud server network via acommunication network, the device comprising: a processor; and a memoryconnected to the processor and storing instructions relating to one ormore software program products, wherein the processor is configured toexecute the instructions for the one or more software program productsto perform operations, comprising: reading, by at least one of thesoftware program products, data from a messaging application; operating,by at least one of the software program products, the messagingapplication, based on the data from the messaging application, whereinoperating the messaging application includes analyzing and sortingmessages; and using, by at least one of the software program products,one or more artificial intelligence models trained using data collectedfrom a plurality of users, to at least one of: (i) operate one or moreapplications for assisting a user to perform tasks or (ii) use the oneor more applications independently of the user for performing the taskson behalf of the user, wherein the tasks include all of: providing oneor more draft answers to queries included in inbound messages,automatically to the user via the applications, comprising: analyzingthe inbound messages, using at least one of the artificial intelligencemodels, based on content of the inbound messages; and organizing anagenda for the user, using at least one of the artificial intelligencemodels, comprising: scheduling a unit of time and a place for one ormore activities in the agenda based on available time in a usercalendar, wherein the cloud server network repeatedly updates artificialintelligence files for a number of tasks, and makes the artificialintelligence files available to the software program products to operatewith the device.
 22. The personal computing device of claim 21, whereinat least one of: (i) providing the one or more draft answers to queriesor (ii) organizing the agenda, is further based on a prior input by theuser for at least one of the one or more applications.
 23. The personalcomputing device of claim 22, wherein the at least one of: (i) providingthe one or more draft answers to queries or (ii) organizing the agenda,at least partially replicates the prior input by the user using at leastone of the artificial intelligence models.
 24. The personal computingdevice of claim 21, wherein at least one of: (i) providing the one ormore draft answers to queries or (ii) organizing the agenda, is furtherbased on using crowd sourced data with the at least one of theartificial intelligence models.
 25. The personal computing device ofclaim 21, wherein at least one of the software program products analyzesmessages from the messaging application and selects an indication soundautomatically based on one or more of: contents of the messages, senderof the messages or one or more recipients of the messages.
 26. Thepersonal computing device of claim 21, wherein at least one of thesoftware program products provides a route calculator via a mapapplication that provides a combined route of driving and walking basedon data from all of: (i) the email or calendar application, (ii) asearch application, and (iii) an Internet browser.
 27. The personalcomputing device of claim 21, wherein the one or more applications areoperated with using the one or more artificial intelligence models toproduce the agenda.
 28. The personal computing device of claim 21,wherein the agenda comprises a plurality of activities and organizingthe agenda for the user comprises determining an optimum total fit ofall of the combined plurality of activities in the agenda for a unit oftime.
 29. The personal computing device of claim 21, wherein providingthe one or more draft answers to queries included in inbound messagescomprises individually determining draft responses to the queries, andguiding the user through the responses.
 30. The personal computingdevice of claim 21, wherein the inbound messages comprises an email, andwherein analyzing the inbound messages comprises associating a meaningto the email based on semantic content in the email.